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EXECUTIVE SUMMARY 


Building on decades of commitment, work, and 
investment by President and Mrs. Bush to improve 
educational outcomes for all students, the George W. 
Bush Institute began the Middle School Matters (MSM) 
initiative in 2010 with the goal of increasing the number 
of students who are prepared for high school and post- 
secondary success. The initiative accomplishes this by 
drawing upon evidence-based research to develop 
practical tools and engaging support opportunities 

for middle grade campuses, allowing research-based 
practices to be brought to life in classrooms across 

the nation. The heart of the initiative is the deliberate 
connection between research and the instructional 
practice of educators. More specifically, MSM turned 
high-quality research into 
actionable strategies for 
districts, schools, and teachers 
to use to improve reading, 
writing, and math instruction 

— and to improve use of data 
systems to identify students who 
are atrisk of dropping out. 


MSM‘s two objectives are 
first to compile the proven 
principles, practices, and 
strategies that can improve 
student success and second 
to help educators actually 
learn, adopt, and master these strategies. In order to 
meet these objectives, MSM was designed with expert 
researchers and practitioners and then implemented 
directly in schools via three cohorts. 


This report continues the Bush Institute’s commitment to 


“MSM stood out over other PD 
because of the title: Middle School 
Matters. We rarely came across 
stuff that was specifically for middle 
schools. The whole premise is fixing 


things for students before they get 
to high school. There used to be no 
PD to fix the middle. Now we have 
researchers and information tailored 
to us.” 
(District Administrator) 


advancing educational outcomes broadly by sharing 
learnings to date about creating and implementing 
MSM. The report’s goal is to help school leaders, 
policymakers, philanthropists, and others learn from our 
experience to better understand: 

- The power and promise of connecting 
high-quality research fo instructional practice; 

- The importance of rigorous (and supported) 
implementation when seeking to improve 
teacher or leader performance; and 

- What to look for—both in terms of challenges 
and solutions—when implementing school-based 
improvement initiatives. 


Most teachers welcome opportunities to learn about 
and master research-based strategies to improve 
outcomes for students, 
especially when they can learn 
directly from experts who can 
model these strategies. Yet 
finding these proven techniques 
and practices can be difficult. 
Professional development (PD) 
for teachers in schools tends 

to adopt “flavor-of-the-month” 
techniques, whereby districts 
incorporate a fragmented 
sequence of learning that is 
ineffective. |? 


Effective professional 
development both exposes teachers to a logical 
sequence of research-based and job-embedded 
techniques and helps with the implementation of those 
techniques in complex classroom environments*. For 
example, when educators apply the proven strategies 
identified and taught by MSM, teacher performance 


'DarlingHammond, L. et al. 2009. “Professional Learning in the Learning Profession: A Status Report on Teacher Development in the United States and Abroad.” National 


Staff Development Council. 


“Blase, K. A., Fixsen, D. L., Sims, B. J., & Ward, C. S. 2015. Implementation science: Changing hearts, minds, behavior, and systems to improve educational outcomes. 


Oakland, CA: The Wing Institute. 


° Garet, Michael S., et al. "What makes professional development effective? Results from a national sample of teachers.” American educational research journal 38.4 


{2001}: 915-945. 


* As noted in the observations and interviews conducted as part of the program implementation evaluation. 
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and student engagement increase’. However, there is a 
significant difference between learning the basics of a 
new strategy and consistently, effectively applying that 
strategy in a school or class. As implementation science 
and extensive experience suggests, familiarity and 

will are not enough for most people to fully adopt and 
master new approaches. 


Thus, based on our experience and learning through 
MSM, it is clear that successful transformation of teacher 
and leader performance requires extensive, systemic 
attention to high-quality implementation, including: 

- Following initial training sessions with 
opportunities for participants to practice and 
receive feedback on the new strategies; 

- Developing implementation processes or 
structures that provide continuous emphasis 
on the core principles, 
learning and 
improvement, 
problem-solving, 
and accountability 
for all involved; 

- Ensuring the new 
initiatives are focused 


“We don’t expect our students to be 
able to master something after we 
teach a concept to them once. | think 
as teachers, we learn the same way.” 
(QI Tolal=19)} 


implementation. MSM has provided an accessible, 
highly valuable suite of tools grounded in high-quality 
research to middle grades educators. While the 
progress among MSM participants is encouraging, 
successful implementation has been challenging. 

As a result exciting successes in some schools have 
been mixed with insufficient progress in others. This 
report seeks fo use the lessons of MSM’s design and 
implementation to inform and support the efforts 

of others seeking to improve student outcomes by 
implementing school-based programs. 


INTRODUCTION AND OVERVIEW 


The Genesis of Middle School Matters 

In the last 20 years, untold time, effort, and money have 
been spent to improve student 
outcomes—trom billions spent 
to turn around failing schools to 
raging debates about curricula 
to experimental initiatives like 
Race to the Top. Yet despite all 
this activity, there are still too 
few success stories of school 


on just two or three 
new strategies and 
align with or replace 
other initiatives 
occurring at the 
same time; and 

- Measuring progress, 
both in terms of 
implementation and 
adoption as well as 
ultimate outcomes, and 
using that data to drive 


further improvement. 


“The feedback and PD was helptul. 
They sat down and talked to us 
individually about what we did well and 
what we could do better. The feedback 
was very tangible about what we could 
do. We talked specifically about the 
thought process we could go through 
with the scholars.” 

(Teacher) 


improvement initiatives that 
systematically improve student 
performance. Fortunately, the 
sector is gradually learning 
what works. 


President and Mrs. George 
W. Bush bring a decades-long 
commitment to the success of 
students. That commitment has 
always been guided by what 
high-quality research indicates 


works best for student learning and growth, teachers’ 


In summary, achieving better educational outcomes for 
students requires both proven practices and effective 


instruction, and educational leadership. The work of the 
Bush Institute operates under the same philosophy. How 
can high-quality research help to improve outcomes for 
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all students? 

With this in mind, the Bush 
Institute convened an 
extensive array of education 
experts—from researchers to 
practitioners to policymakers— 
fo create a 

research-based initiative to 
improve education outcomes 
for students. What emerged 
was further clarification of three 


fundamental realities. 


First, according to Sandy 
Kress, an early architect of the 
initiative, “the middle grades 
were the last best chance that 
we have to get youngsters 

on the right path [toward 
academic and career success]. 
It’s not necessarily early 
enough, but later is too late.” 

A lot happens for students in 
the middle school years, both 
personally and academically. 
For example, for the first time: 
they are taught by multiple 
teachers on separate subjects, 
receive more homework, 

have higher expectations for 
students to self-manage, there 
are tangible consequences 

for getting off track, and much 
more. Itis uniquely challenging 
for teachers and principals 

to help students successfully 
navigate their social-emotional 
and academic growth. Despite 
extensive research to this 
effect, public and private 
attention to these pivotal years 


“A lot of models around school reform 
are hatched by people based on an 
intuitive sense of what will work with 
kids. Education often floats around 

in a realm of grayness that seems 


intuitively plausible. In our early 
discussions, the group was strongly 
oriented towards using as much 
evidence of effectiveness as we could 
bring to bear on the issue.” 

(Mark Dynarski) 


“Schools under NCLB and ESSA 
have been asked to use evidence- 
or research-based practices. It’s 
a reasonable request and most 
stakeholders and parents say, ‘Of 
course!’ But there’s a problem. When 
you talk to key decision-makers — 
whether district administrators, 
principals, or teachers —about what 
they actually do, it becomes clear that 
they mostly rely on the set of practices 
they’ve been using over time. Some of 
these practices are quite effective. And 
whether they have research behind 
them or not, they’ve served as great 
extensions that help them do their job 
as teachers. However, the research 
base in education has outpaced the 
practice and implementation. We 
‘know’ a lot more than we ‘do.’ There’s 
a lot of knowledge out there that isn’t 
been utilized in schools.” 

(Sharon Vaughn) 


tends to trail behind that given 
to elementary and high school 
efforts. While it makes sense to 
ensure students get a great start 
in elementary and are later 
ready for college and beyond, 
middle schools are also a 
critical piece of that process. 


Second, rapidly expanding 
best-practice research findings 
are not readily in the hands of 
educators. In the last 10-15 
years, rigorous research has 
made substantial progress 

in identifying the specific 
instructional strategies that 
drive student success. As 
Kress noted, “a good bit of the 
research that had been done 
in the previous decade (largely 
under NIH or IES grants) said 
something about adolescents 
or youngsters in these [middle 
school] grades. So we had a 
corpus of research that was 
newly available.” Yet the 

vast majority of education 
practitioners—administrators, 
school leaders, and teachers— 
are unaware of or not using 
these proven strategies. ° 
There are many reasons for 
this gap. Researchers typically 
don’t have the time, ability, 

or incentives to ensure their 
insights get all the way into 
the classroom. And educators 
are not often prepared, nor do 
they have the time, to explore, 
interpret, and apply the latest 


5 Cohen, J., McCabe, E., Michelli, N., Pickeral, T. 2009. “School Climate: Research, Policy, Practice, and Teacher Education.” Teacher College Record. 111 (01). 180-213. 
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research on their own. Furthermore, most purveyors 

of effective teaching and school leadership—whether 
schools of education, private consultants and trainers, or 
state and district offces—don’t bridge this gap between 
research and practice very well. As a result, educators 


are left to sort it out on their 
own based on their individual 
training, experiences, 
situations, and style. 


As the growing held of 
implementation science 
demonstrates, even when we 
know what works, it is very 
difficult to get individuals and 
systems to accept, adopt, 
master, and consistently apply 
new concepts. Reaching full 
implementation takes much 
more than how-to manuals, 
one-off presentations, or 
periodic training seminars. 
Rather, it requires extensive 
learning and practice, 
feedback and coaching, 
incentives and accountability. 


In the field of healthcare, 
where the understanding 
and use of implementation 
science is increasingly robust, 
research has shown that 
most implementation efforts 
result in outcomes that are an 
average of 12 times better 
than those resulting from 


typical implementation techniques.” And in education, 
recent research shows that teachers’ mastery and 


“Education doesnt live in the same 
kind of world that most enterprises 
do—it’s just not research-based. 

mi =10) 6) (<M ATA =me) 6/10) 6)4 Ym 6216) 62) 
have views. But people don’t make 


decisions about money and programs 
based on data and research. We're still 
in a ‘pre-Flexner’ world in education.” 
(Sandy Kress) 


“When we came back from MSM in 
June, we felt like just bringing the 
teachers together wasn’t enough. 

| have learned that the one shot 
approach to PD does not work without 
the follow-up. It was my thought to 
oO (oM am tal-merey-Tele]ialem e)(e-meig lem tare 


MSM leadership team. If we didn’t 
have the push and the support piece 

in place, | would have had several 
teachers who would have said, ‘It’s just 
one more thing,’ and not put it in place. 
Everyone is held equally accountable.” 
(Principal) 


as well.” 


use of new techniques reaches only 5 percent when 
they're simply taught the concept and discuss it in 
theory.® Unfortunately, this mode of implementation 
is alltoo-typical in educational PD — and teachers 
recognize this. As one school leader involved with 


MSM noted, “professional 
development without follow- 
up is malpractice.” Instead, 
when educators learn new 
techniques through a more 
robust approach to PD— 
one involving instruction, 
observation, practice with 
feedback, and ongoing, 
in-class coaching—depth of 
implementation reaches 95 
percent. 


With these gaps in mind, the 
Bush Institute and the experts 
we convened reached a 
compelling conclusion—more 
high-quality research is 
needed about what works 

in educational instruction 

and leadership, and we must 
ensure that existing and future 
research guides the educators’ 
work and efforts in practice. 
With a historical eye on the 
transition healthcare has made 
over the last century based on 
the insights from the Flexner 
Report, the group readily 
agreed that the there was a 
significant opportunity 


to catalyze education’s transition from craft to science 


°Vernez, G., Karam, R., Mariano, L. T. & DeMartini, C. 2006. "Comprehensive school reform: The implementation gap.” Santa Monica, CA: RAND Corporation. 

” Fixsen, D., Naoom, S., Blasé, K., Friedman, R., Wallace, F. 2005. Implementation Research: A Synthesis of the literature. University of South Florida. 

® Joyce, B. & Showers, B. 2002. Student Achievement Through Staff Development. ASCD. 

° Flexner, A. 1910, “Medical Education in the United States and Canada: A Report to the Carnegie Foundation for the Advancement of Teaching." Bulletin No. 4., 
New York City: The Carnegie Foundation for the Advancement of Teaching. 
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The Program Model 
The Middle School Matters initiative incorporated three core elements: 

- Arepository of instructional and leadership principles, 
practices, and strategies designed to help teachers and 
leaders implement the insights from high-quality research. 

- Asummer conference that introduces participating schools 
and districts to these strategies and helps them develop 
customized implementation plans. 

- Professional development, provided onsite and virtually 
at various points throughout the year, to help educators learn 
and adopt the strategies they have chosen to implement. 


MSM Resources—Principles, Practices, and Strategies 


Following the decision to focus on connecting high-quality education 
research with educators’ efforts in schools, MSM engaged leading 
scholars in both instruction (including reading, writing, math, cognitive 
science) and student supports that enhance learning (including dropout 
prevention, school climate and culture, student behavior and motivation, 
and extended learning time) to develop the core content of the MSM 
program.'° Each scholar was tasked with distilling the latest, most relevant 
research about his/her topic in order to identify the principles and 
practices that have been proven to improve student outcomes in that 
area. As Beth Ann Bryan, an original advisor to the MSM team, recalls, 
“there was a lot of pressure to include ideas in our materials that didn’t 
have a lot of evidence, including topics like school leadership"'.” But both 
the Bush Institute and the full group of scholars and practitioners agreed 
that they would only draw from studies employing the most rigorous 
research methodologies, mirroring standards used by the National 
Institutes of Health (NIH) and the Institute of Education Sciences (IES). 
This is an important point of distinction for Middle School Matters—all 
recommendations, strategies, and materials are based on scientifically 
proven methods for improving student outcomes. The result of this work 
is a series of guiding principles and practices in each topic area, as 
summarized in Table 1 below. 


Following extensive collaboration with other scholars and practitioners 

to clarify and define these insights (see the Making Research Accessible 
section), the MSM experts translated the research into educator-+friendly 
strategies, principles and practices. All of these materials have been made 
available via the Middle School Matters Field Guide (Field Guide) and on 
the initiative’s website (www.greatmiddleschools.org). These resources not 
only provide a central source for the best research regarding the drivers 

of middle school success, they are replete with overviews, how-to guides, 
toolkits, multimedia examples, FAQs, and more so that educators can 
easily turn research insights into reality for students. 


'©The list of scholars can be found here: https://greatmiddleschools.org/download-view/cover-and-introduction/ 


2010-2011 


Expert researchers, policymakers, and 
educators met to identify MSM's focus 
and the core instructional strategies. 


2012 

Researchers and educators 
collaborated to build the first edition 
of the Field Guide and tools. Bush 
Institute partners with the Meadows 
Center for Preventing Educational Risk 
(MCPER) to complete the materials 
and to design and implement the 
cohort model. 


2013-2014 
The first MSM summer conference held 
in June 2013 followed by the launch of 
the first cohort. 


2014-2015 

Second summer conference held and 
Cohort 2 launched. Schools select 
fewer focus areas and the MSM team 
made additional site visits to schools to 
improve adoption. 


2015-2016 

The third summer conference is 
held. Cohort 3 schools selected 
fewer (2-3) strategies in order to 
increase implementation. MSM 
team conducted four PD site visits per 
school along with ongoing advising 
and support. Districts participated 
alongside schools to facilitate scale 
and sustainability 

over time. 


2016 

MSM published the second edition of 
the Field Guide, along with the release 
of an updated website with new 
toolkits and support materials. 


'| GWBI also focuses on school leadership policy and research: http://www.bushcenter.org/explore-our-work/developingeaders/alliance-+to-reform-educationleadership. html 
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TABLE 1: MSM PRINCIPLES 


RESEARCH-BASED INSTRUCTION 
READING WRITING MATHEMATICS COGNITIVE SCIENCE 


1. Establish schoolwide 
practices for enhancing 
reading for understanding 
in all content area 
instruction. 


2. Teach word-meaning 
strategies within content 
area classes. 


3. Activate and build 
appropriate background 
knowledge for 
understanding text content. 


4. Teach students to use 
reading comprehension 
strategies while reading 
complex text. 


5. Provide intensive 
reading interventions to 
students with reading 
problems. 


6. Guide students during 
textrelated oral and 
written activities that 
support the interpretation, 
analysis, and 
summarization of text. 


7. Maximize opportunities 
for students to read and 
connect a range of texts. 


8. Organize students into 
collaborative groups for 
reading tasks. 


9. Discontinue using 
practices that are NOT 
associated with improved 
outcomes for students. 


1. Establish consistent 
schoolwide practices for 
using writing as a tool to 
support student learning 
in all content areas. 


2. Explicitly and 
systematically teach 
students the processes, 
knowledge, and skills of 
effective writing. 


3. Establish word 
processing as the 
common medium for 
student writing. 


4. Assess and monitor 
student writing to improve 
instruction and identify 
students who require 
more intensive writing 
instruction. 


5. Provide extra 
assistance and instruction 
to students who 
experience difficulty 
learning to write. 


6. Discontinue using 
practices that are NOT 
associated with improved 
outcomes for students. 


7. Improve teacher 
capacity to teach writing 
and use it as a tool for 
learning. 


1. Establish schoolwide practices for 1. Distribute presentation, 
enhancing mathematics understandirg practice, and testing over 
within relevant content area instruction. time. 


2. Ground ideas in active, 
engaging experiences. 


2. Use a universal screener to identify 
students at risk for mathematics 
difficulties and to determine 
interventions to provide these at-risk 
students. Monitor the development of 
mathematics knowledge and skills of 
identified students. 


3. Provide timely, 
qualitative feedback on 
students’ learning activities. 


4. Encourage the learner 
3. Help students recognize and to generate content. 
expand their understanding of number 
systems beyond whole numbers 
to integers and rational numbers. 
Use number lines as a central 
representational tool in teaching this 


and other fraction concepts. 


5. Select challenging tasks 
that require explanations, 
reasoning, and problem 
solving. 


6. Design curricula, tasks, 
and tests in different 
contexts, media, and 
practical applications. 


4. Develop students’ conceptual 
understanding of mathematics and 
provide ample opportunities to 
improve procedural fluency. 

7. Promote self-regulated 
5. Provide explicit and systematic learning. 


instruction during intervention. 


6. Instruction should include strategies 
for solving word and algebra 
problems that are based on common 
underlying structures. 


7. For students who struggle in 
mathematics, instruction and 
intervention materials should 

include opportunities to work with 
representations of mathematical ideas. 
Teachers should be proficient in the use 
of these representations. 


8. Establish a schoolwide plan 

to identity and improve teachers’ 
mathematical content knowledge and 
pedagogical content knowledge. 


9. Discontinue using practices that 
are NOT associated with improved 
outcomes for students and teachers. 
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STUDENT SUPPORTS THAT ENHANCE LEARNING 


CTIIDENIT REHA\ > [ 
| STUL CIN|I DEMAVIVUI 


mae: | 


1. Consistently teach, model, and recognize appropriate 
and positive academic and social behaviors across all 
classrooms. 


2. Provide classroom instruction in self-monitoring and 
regulation, academic organization and study skills, goal 
setting, persistence, and healthy behaviors. 


3. Establish processes for identifying problem behaviors 
early, diagnosing their causes, identifying effective 
interventions, applying the interventions at the scale and 
intensity required, and monitoring their effectiveness. 


4. Make the value of schooling personal. 
5. Create a sense of belonging for all students. 


6. Connect students’ academic success to effort. 


DROPOUT PREVENTION 


1. Use data systems to identify students who are at risk of 
falling off the path to high school graduation. 


2. Assign adult advocates to students who are at risk of falling 


off the path to high school graduation. 


3. Provide academic support and enrichment to improve 
academic performance. 


A. Implement programs to improve behavior and social skills. 


5. Personalize the learning environment and instructional 
process. 


1. Create a “can do” school culture marked by a shared mission 
among the staff members that centers on academic achievement and 
shared belief that they can collectively enable students to succeed. 


2. Create a school environment in which mutually supportive 
relationships between students, teachers, and parents can develop. 


3. Engage in schoolwide efforts to increase student attendance, 
promote positive behaviors, and increase student effort (where 


needed). 


A. Focus the school-family partnership on communicating to students 
the importance of high academic and educational aspirations 

and showing the steps that need to be taken to actualize these 
aspirations. 


5. Conduct studentneed and asset analyses and select community 
partners and supports based on student need. Design and manage 
a plan to link community supports to success in school and use 
common metrics to gauge their impact. 


1. Align the Extended Learning Time (ELT) program academically 
with the school day. 


2. Maximize student participation and attendance. 
3. Adapt instruction to individual and small group needs. 
4. Provide engaging learning experiences. 


5. Assess program performance and use the results for program 
improvement. 


It may be helpful to provide an illustration of the connection between the MSM principles, practice, and strategies, as well 
the available resources that help educators apply these concepts. As noted in Table 1 above, the fourth reading principle 
states: “Teach students to use reading comprehension strategies while reading complex text.” '? This general concept is 


supported by two recommended practices: 


1. Instruct students to generate questions while reading to build comprehension. 
2. Instruct students to generate main ideas at regular intervals in a text. 


The Field Guide further elaborates on the first practice by introducing the instructional strategy known as “Leveled Questions” 

in which teachers encourage students to ask three types of increasingly nuanced questions to increase their reading 
comprehension. In addition, the Field Guide links to a toolkit to help teachers learn how to use leveled questions in the classroom. 
The toolkit includes online instructional modules, presentations, videos that demonstrate the strategy in action, and supplemental 
lesson and student materials for teachers’ immediate use. Finally, the site provides a complete research bibliography for those 
interested in further understanding of the evidence that supports the use of these principles, practices and strategies. Similar 
resources are available for many of the MSM topics and principles. 


'2 MSM Field Guide: Reading and Reading Interventions, pA-12. Available at: https://greatmiddleschools.org/download-view/reading-and-reading-interventions/. 
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MSM Program Model 


In addition to compiling the proven principles, practices, 
and strategies that can improve student success, the 
second essential objective of MSM is to help educators 
actually learn, adopt, and master these strategies. 

To do this, the program model included extensive 

PD, delivered by the MSM team of researchers and 
coaches. In addition to the MSM website described 
above, the PD delivery model included three elements: 

- Summer Conference — A multi-day opportunity for 
teachers and administrators to learn about the 
various MSM principles and develop tailored 
plans to implement a few of those principles in 
their own schools. 

In-Person Training —Two-day training sessions 
provided by MSM researchers throughout the 
academic year to train teachers on the MSM 
principles selected by the school (increased to four 
visits by Cohort 3). 


The delivery of the program truly began with the summer 
conference, which had multiple objectives: 

- Provide an introduction to specific principles, 
practices, and strategies and how they apply to 
the goals of the schools and districts; 

- Introduce ongoing access to the MSM resources 
available via the website; 

- Introduce the concept and merits of research- 
based practices to improve student outcomes; 
Moderate schools’ self-evaluation of needs based 
on student achievement data; 

Encourage the adoption and implementation of 

these practices by schools and districts based on 

the participants’ needs and objectives; and 

- Build relationships between school leadership and 
the MSM coach. 


Following the conference, the MSM team provided 
in-person training to each cohort member during the 
school year. The PD was 


Sessions including 
lectures and 
presentations, modeling 
of the strategies with 
students, and some 
observation and 
coaching of teachers 
using the strategies in 
their own classrooms. 

- Coaching and Support 
On-call support by MSM 
coaches to answer 
questions, identify 
additional resources, 
provide additional 


training via webinars, etc. 


Schools applied to be part 

of MSM, and once selected, 
sent a team of school leaders, 
instructional coaches, and 


“The summer conference helps you to 
build up that trust with the coach and 
researcher. You get to sit right next 

to them, so it’s not something that’s 
coming from out of nowhere. They 
can bring in their suggestions [to the 
school], and we can take it as they 
really care and they really want the 


best. The summer conference helped 
build that relationship. Next year | 
would add a science and a history 
teacher to the leadership team that 
attends the conference to have their 
buy-in as well.” 

(Instructional Coach) 


provided primarily by an 
expert researcher who has 
both deep familiarity with 

the underlying research and 
extensive experience applying 
the instructional strategies in the 
classroom. The first session was 
primarily an introduction and 
planning session-sharing the 
MSM platform with the entire 
school, explaining why the 
school team chose the specifc 
principles/practices that it 

did, and finalizing plans for 
subsequent training sessions. 
The remaining sessions 
typically lasted 2-3 days 

and involved several hours 

of professional development 
along with the opportunity 

for teachers to either see that 


faculty to the summer conference to learn about the 
MSM principles and develop an implementation plan. 
In conjunction with their MSM coach, each school team 
then selected specific principles/practices from the 
MSM Field Guide on which to focus in the year ahead. 
This choice was based on a variety of factors, including 
student performance data and needs, alignment 

with ongoing initiatives, and the availability of MSM 
resources. 


practice modeled in their own classroom or to receive 
feedback after they try to implement it themselves. 


The presenter demonstrated how to apply the strategy 

in practice, first with the educators in attendance and 
then with a group of students at the school. At the end of 
the session, the expert often visited classrooms, either to 
further demonstrate the strategy in different settings and/ 
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or to observe and provide 
feedback to a teacher trying 
to implement the new strategy 
with his/her own students. The 
next day, this individualized 
coaching continued. In 
addition, the teachers, the 
expert researcher, and the 
MSM coach reviewed training 
provided on a different 
strategy in the previous site 
visit to check on progress and 
address questions that arose 
as teachers implemented the 
strategy over time. 


Between in-person training 
sessions, a MSM coach 
played a key coordination 
and support role throughout 
the delivery of MSM PD. The 
coach provided additional 
materials, tools, modeling, 
and feedback to support 
teachers’ adoption of 
instructional strategies. The 
coach proactively shared 
additional research materials 
with principals or instructional 
coaches to distribute to their 
broader leadership team and 
teachers as needed based 
on regular check-ins with 
demonstration schools. At 
the end of the school year, 
the MSM coach worked with 
the school’s leadership team 


“They (MSM team) would come in 
and meet with us about where we 
were with our implementation. That 
kept us accountable so that we as 
a leadership team could realize 
where we haven't met our goals 
so we can go into our PLCs and 


determine where we're going to put 
the strategies in. It’s an opportunity 
to keep the teachers on track and 
remind them when we haven't seen 
something we need to see.” 
(Instructional Coordinator) 


“Just being in contact with the coach 
and researcher has been helpful to 
me. Being able to contact them to ask 
those questions is helpful... | can talk 
to him and email him. For example, | 
was preparing for a presentation on 
lesson plans, and | contacted (MSM 
Coach) about gradual release lesson 
plan templates, and he looked at that 
and gave us some options of lesson 
plan templates we could try out. It’s 
things like that they are there for.” 
(Instructional Coach) 


to discuss progress made 
throughout the year and 
provide guidance regarding 
the school’s implementation 
and ongoing planning efforts. 


In the first two years of 
implementation, MSM worked 
at the school-level. In an 

effort to facilitate both scale 
and long-term sustainability, 
MSM expanded the model 
during the 2015-2016 school 
year to include a second 
middle school in a district as 
well as district leaders. One 
school was selected as the 
demonstration school, which 
received direct engagement 
from the MSM team. The 
second school was selected as 
the pilot school. District officials 
and pilot school staff were 
invited to attend the site visits 
at the demonstration to learn 
about the MSM strategies and 
see them in action. The district 
officials, rather than the MSM 
coach, then provided direct 
ongoing support to the pilot 
school. 


“There needs to be more accountability. On walk-through day or on [site 
visit] day, we are held accountable, but as far as general, every day 
[accountability], there’s nothing. | think there needs to be something. There 


are teachers that are totally bought in, and then there are teachers that are 
only going to do it the week of and for the walk-throughs.” 
(Teacher) 
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LESSONS LEARNED 


There are a number of lessons learned from the six years 
of development and delivery of Middle School Matters. 
The most important and salient of these are directly 
related to the initiative’s original objectives—increasing 
accessibility to the existing research on what works for 
middle school success and determining how to turn 

our knowledge about what works to improve student 
outcomes into practice in schools and classrooms. 

In addition, the process of connecting researchers 

and educators to improve student outcomes yielded 

a number of additional lessons that are relevant 

for anyone working on school-based improvement 
initiatives. 


Making Research Accessible 


It is easy to blame educators for not doing a better 

job of learning about, adopting, and applying best 
practices as determined by rigorous research, but there 
are a number of complex reasons why this does not 
happen naturally. A fundamental challenge is that the 
basic interests and outlooks of education practitioners 
and academic researchers are very different, making 
it difficult for knowledge or experience from one group 
to transfer to the other.'* Furthermore, leaders from the 
two professions rarely interact, making it hard for each 
group to connect with, understand, and learn from 

the other. As a result, researchers too often assume 
that their statistical findings can be easily understood 
and translated into action by teachers. Teachers have 
limited opportunities to learn from and ask questions 
of researchers about their work - much less provide 
feedback to those researchers - to ultimately improve 
the nuance, precision, and practicality of researchers’ 
insights. 


In addition, the body of research is expansive and 
sometimes contradictory, making it difficult for any 
one teacher or administrator to know what findings 
deserve attention. Even when a research study has 
conclusive findings, those findings are rarely, if ever, 
generalizable to all school contexts given the many 
disparate situations that exist in a particular school or 
district. Finally, research increasingly involves more 
and more specialization, meaning that most individual 


researchers’ depth of expertise decreases as the range 
of topics or issues grows. 


MSM‘s initial approach to bridging this gap has been 
crucial to the success the program has subsequently 
achieved. As described above, this began with an open, 
solution-oriented discussion among education experts. 
The key is that this work involved a range of different 
types of experts—experienced researchers representing 
multiple issue areas (e.g. reading, math, school climate, 
etc.), policymakers, seasoned administrators, and 
experienced teachers. Getting this full set of perspectives 
in the same room for multiple discussions was crucial 

as everyone had an equal opportunity and equal 
responsibility to advance the dialog. 


The next step involved gaining group consensus as fo (a) 
the potential value of using high-quality research findings 
to drive school improvement and (b) what qualifies as 
rigorous research. Alignment on these guiding principles 
created both clarity and objectivity about what would 
and, importantly, what would not be included in the 
initiative. Without agreement on these criteria, similar 
discussions routinely devolve into arguments based on 
personal preferences rather than proven results. In this 
case, however, topics which met the high bar of research 
rigor were quickly identifed and focused the initiative. 


Once the group identified the topics that met the 
research criteria, experts in each area took the lead in 
identifying the most relevant principles and practices that 
would improve student outcomes to include in the Field 
Guide. In doing so, each contentarea expert attempted 
to address a basic question for his or her chapter of 

the guide: “What should the field know and do about 
this topic?” Through considerable work, each author 
developed an initial list of principles and practices the 
feld had tested and proven to improve student success. 
Each principle came with a fully-documented set of high- 
quality research to justify its inclusion. 


With these drafts of the Field Guide in place, the 
researchers shared their recommendations both with 
other researchers and with teachers. Through several 
rounds of iteration, each group asked questions and 
gave feedback that increased the clarity and practicality 
of the guides. In some cases, changes involved simply 


'S Shavelson, RJ. 1988. “Contributions of educational research to policy and practice: Constructing, challenging, changing cognition.” Educational Researcher, 
17 (7), 4-11; 22. Trubowitz, S. & Longo, P. 1997. How it works: Inside a school-college collaboration. New York, Teachers College Press. 
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removing academic jargon and re-casting the 
principle in terms more familiar to teachers. In other 
cases, teachers worked with researchers to ensure the 
research finding was actually something that could be 
applied in practice. The experts at the MCPER played 
a pivotal role here, working extensively to identify 


instructional strategies that exhibited the sometimes 
esoteric principles covered in the research. As Christy 
Murray at the MCPER notes, “The researchers 
themselves didn’t understand the level of detail we 
needed to get to for teachers to apply things 

in classrooms.” 


INAISY MMM -To cola dol-lacdel ale MocoLo(olal-1m oLo) Mul-Mami-l(e M10 o(-Molste]ol-1eMelaleMolg-x-lalrelifolaree(-n4-1elel-Lol oat) 
researchers and, over the last several years, incorporated the feedback of hundreds of teachers to develop 
clearer research summaries, additional training materials, videos showing the strategies in action, and toolkits 


to apply the strategies with students. This experience was new for everyone involved, but highly gratifying 
and beneficial. The result is a website that provides a comprehensive and rigorous yet accessible and 
practical set of resources about how to improve middle school student success. The Field Guide was updated 
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Mastering a New Strategy 


Whether you are an educator or not, mastering and 
consistently applying a new behavior is hard. After all, 
most New Year's resolutions fall by the wayside only a 
few days or weeks into the New Year. It’s no different 
when educators—whether 
newly minted or seasoned 
veterans—are introduced to 
anew approach to teaching 
or leading. Even the most 
optimistic and committed 
will struggle to master a 
new approach. In light of 


insights drawn from MSM'’s (Mark Dynarski) 
implementation. 
¢ First, educators are hungry for deeper, 
longer-lasting professional development 
support. 
Most educators are looking for high-quality PD 
opportunities to deepen their practice. But, quality 
offerings are limited, and educators often struggle to 
put into practice what they learn in a one-day seminar. 
Cohort 1 schools shared this feedback when the in- 
person PD initially consisted of only one daylong site 
visit. In light of this, in-person support was gradually 
increased over the three cohorts to become four multi- 
day visits throughout the year accompanied by ongoing 
webinars and offline support by Cohort 3. This year-long 
support was highly appreciated and highly valued. 
It provided teachers with an initial introduction to the 


“That this process [connecting 
researchers and educators] is not 
more often used is a pity to me. This 


dance between research and practice 
should be mutually reinforcing like 


this reality, there are several this.” 


key principles, time between sessions to try out the 
corresponding strategies, and subsequent opportunities 
to ask questions and learn more about how to put 

the principles in practice. Of course, even though this 
level of ongoing PD is much more extensive than most 
teachers receive, MSM participants agreed that - time 
and budget permitting - it would 
be even better if MSM or any 
other PD initiative could provide 
even more extensive support. 


¢ Second, individual 
improvement requires 
ongoing, systematic 
follow-up. 

When left to their own devices 
(even with multiple rounds of 
support from MSM experts), 
only a small percentage of 
teachers truly mastered the new strategies by the end 
of the school year. However, when schools created 

or applied a formal system and structure to roll out, 
reiterate, and sustain new initiatives, more than half of 
participating teachers gained some level of mastery 
in the same time period. Why? The obvious answer is 
that continual reminders, additional opportunities for 
learning (especially from respected peers), ongoing 
encouragement, and even alittle bit of pressure all 
contribute to changing one’s behavior regardless of the 
setting. In the most recent implementation year, three 
of the participating schools used such systems to drive 
success. And while each of these schools differed in its 
approach and process, all three systems included: 
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A strong principal committed to high-quality 
instruction. These principals were visible, 
adamant, and consistent in their leadership, 
ensuring that everyone in the school recognized 
that research-based teaching was one of (if not 
the) school’s top priorities. They publicly 
celebrated successes, provided ongoing 
encouragement, and even implemented negative 
consequences when necessary. 

A strong and distributed leadership team. These 
teams were comprised not only of administrators, 
but of both new and experienced teachers, 
instructional coaches, and non-academic support 
personnel. All team members shared the 
principal’s commitment to bringing 
research-based practices into schools. Just as 
important, they developed and implemented a 
consistent process for providing additional 
instruction and feedback, reviewing progress, 
and ensuring accountability. Each step along the 
way, they focused on both the school as a whole 
and on each individual teacher. 

Finally, the principal 
and leadership teams 
exhibited a sense of 
entrepreneurialism, 
addressing unfamiliar 
or challenging 
situations in a creative, 
proactive, and solution- 
oriented way. While 


they f tl ht ; 
ee strategies on our own.” 
advice and 
resources from the (Teacher) 
MSM coach and 


experts, they took responsibility for success on 
themselves, determining on a daily basis how 
best to implement the MSM principles with fidelity 
in their local level. 


Of course, others have recognized the importance of 
such in helping individual educators improve their own 
performance.’ Yet the reality remains that far too many 
PD efforts—whether homegrown or externally driven— 
continue to ignore these essential realities. PD without 
ongoing instruction, feedback, and follow-up often 
amounts to outputs without outcomes. 


“Repetition was key for us. | know 
they're trying to reach out to as many 
schools as possible, but going (to 

the summer conference) several 


times has really led to our success in 
being able to implement some of the 


Scaling and Sustaining a New Strategy 


The level of investment described above can be 
significant. The challenge of sustaining and scaling such 
an effort is signifcant as well, and there are often a 
number of factors that can inhibit long-term sustainability. 
The implementation of MSM confronted many of these 
obstacles, providing an understanding of what does 
(and does not) work to overcome them. 


¢ Training the trainer can increase scale, but it 
requires its own strategy and investment. 
In the first two years of MSM’s implementation, the focus 
was solely on improving the model-— learning how to 
best train and support educators to use MSM‘s research- 
based practices to improve instructional practice. In Year 
3, the aim was to expand MSM’s impact without hiring 
many additional MSM researchers and coaches. To do 
this, district officials and faculty from a second middle 
school (the pilot school) in a selected district were invited 
to join the delivery of PD to the primary (demonstration) 
school. In bringing these additional audiences into 
the process, essentially as 
observers, the hope was that 
the district officials would learn 
enough to provide ongoing 
support to the second school 
between our in-person visits. 


Unfortunately, that did not 
occur. In hindsight, the gap 
became clear to the MSM 
team. The districts and pilot 
schools needed much more 
support to be effective. They 
needed to become content experts themselves, gaining 
more than an introductory familiarity with the research- 
based strategies and how they should be applied in 

a classroom. Just as important, they also needed to 
become experts in providing the type of PD described 
above, including how to teach and model each strategy, 
provide effective coaching and feedback, and support 
schools’ leadership teams in developing and applying 
their initiative support strategies. And while this would 
indeed involve an entirely additional PD program for 
district staff, itis unfair to expect scale or sustainability to 
occur without it. 


'* As one example among many, see Transforming Schools: How Distributed Leadership Can Create More High-Performing Schools by Bain & Company. 
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¢ Alignment across initiatives matters. 

While MSM was clearly a top priority for most 
participating schools, in no case was it the only 

new or ongoing initiative being implemented at the 
same time. Sometimes the other initiatives involved 
different approaches to improving instruction, in which 
case teachers often became confused, struggling to 
understand which strategies to use when and whether 
some should receive more attention than others. In 
other cases, the other efforts 
were relatively unrelated. But 
even then, at best parallel 


“MSM needs to be a little more hands- 


measuring the right things matters. The problem is that 
just as too many school improvement efforts fail to build 
on solid evidence, too many likewise fail to assess 
implementation and progress. In the case of MSM, while 
the focus was heavily on data-driven design from the 
outset, the need for ongoing assessment was identifed 
several years into the program when a significant 
implementation evaluation was conducted (and from 
which many of this report's highlights emerged). 


Perhaps not surprisingly, 
the evaluation of MSM’s 


initiatives diminished focus 
on learning MSM strategies. 
At worst, they led teachers to 
treat MSM as the “flavor-of- 
the-month,” just the latest in 
along line of initiatives that 


would shortly be supplanted. 


on. When we come back from training 
at [demonstration school], we don't 
really know if we're rolling out the 
training correctly. At the district level, | 


don't have access to the coaches and 
researchers like the demonstration 


implementation found that 
none of the schools were 
systematically measuring 
their own progress in terms 
of teachers’ awareness, 
adoption, or application of 
MSM principles. And while 


While the planning efforts 
at the beginning of the year 
to understand and align 
with other initiatives were 
important, the program 
underestimated the effect the other initiatives would 
have on MSM’s implementation. A recommendation is 
to address this early and often at both the district and 
campus levels to limit the number of initiatives to a bare 
minimum and ensure all involved understand how these 
are related. 


school does.” 


¢ Distributed, team-based leadership protects 
new initiatives against both turnover 
and early opposition. 
Even when everything else is in place, every educator 
has seen how quickly turnover can undermine progress 
in a school. Likewise, in education and elsewhere, 
many turnaround or transformation initiatives face 
resistance from a few vocal individuals, causing others’ 
support to waver. Both of these are important reasons 
to generate on widespread ownership in planning and 
implementation of an improvement effort like MSM. 


¢ Be clear in advance about how you'll know 
you’re succeeding. Then measure, report, 
and adjust accordingly. 

Everyone has heard the old adage, “what gets 

measured, gets done” or its corollary, “what gets 

measured, gets improved.” Either way, the point is that 


(District Coordinator) 


the best school leaders had a 
sense for how many teachers 
were mastering the strategies, 
many were surprised at the 
variation of implementation 
when the independent evaluation was conducted. 
Consequently, both the schools and the program missed 
significant opportunities to provide feedback, learn 
about successes and challenges, and improve along the 
way. In experience elsewhere, this type of ‘measurement 
for learning’ goes way beyond looking at end-of-year 
test results. Rather, it involves deliberate discussions to 
answer questions like: 
What early signals are likely to indicate 
progress toward the ultimate goal of student 
success? For example, how will we know whether 
we are implementing the new approach with 
fidelity; whether that implementation is prompting 
greater student engagement; and whether 
greater engagement is driving student learning? 
- What simple tools can we use to track 
our progress on these indicators in a consistent, 
standardized way? Can we develop an 
observation rubric or a personalized checklist to 
know how well we've mastered the strategy? 
How will we use this data to hold ourselves 
accountable and improve? Who will review the 
data and determine its implications? When and 
how will this occur? 


PAGE 16m 


Lessons Learned: Middle School Matters 


Implementing with Fidelity 


¢ Presume Good Intent. 

It’s all-too-easy for non-educators to assume that poor 
instruction is simply a result of unmotivated teachers. 
This experience suggests the exact opposite. Time and 
again, both anecdotally and in the formal evaluation, 
the vast majority of teachers resoundingly expressed a 
strong desire to help students excel, and the teachers 


other teachers were resolved to keep working at the 
strategies they learned. Teachers’ desire and ability 

to implement research-based strategies increased 
dramatically as they saw progress in themselves and its 
effect on students. 


¢ Focus, Focus, Focus. 
When it comes to learning new strategies, less is 
definitely more. The Field Guide includes the key 


recognized that they need 

to learn and use effective 
strategies to make that 
happen. So while some might 
be hesitant or even resistant to 
a new initiative, most teachers 
are eager fo implement what 
they are taught. 


¢ Success Breeds Success. 
Change management gurus 
and athletic coaches alike 
know this all too well— 

simply knowing what to do 

or believing it is important 

is not enough. Rather, it is 
crucial to engineer, look for, 
and highlight quick wins. 

In this experience, many 
teachers were intrigued 

by presentations about the 
research or even role-play 
examples of how to use the 
strategies. But often they would 
leave the training sessions with 
a heavy dose of skepticism, 
commenting either, “I’ve seen 
things like this before,” or “You 
haven't seen the kids in my 
class.” 


But when the expert visited a 
teacher’s classroom to model 
the strategy and successfully 
engaged his/her students, that 


teacher became more open-minded. That experience 


“There hasn't been any evaluation or 
assessment of how well the program 
is working... no formal process for us 
to look at how effective the strategies 
are. 
(Assistant Principal) 


ELE 


Whether we're talking about eating 
well or exercising or teaching with 
research-based practices, the 
challenge is the same—they only 
work if you keep at it. The difference 
with these kinds of instructional 
practices is that they don’t necessarily 
give you the immediate positive 
impact that you need to sustain 
them like endorphins after exercise. 
Instead, you try the instructional 
strategy out and it’s actually a little 
more than you’re used to. And the 
ultimate outcomes only appear after 
a couple of years. In the meantime, 
many people drop these programs. 
(Sharon Vaughn) 


principles and practices that 
research has shown improve 
student outcomes—dozens of 
things that educators ‘should’ 
be doing. And for those who 
are passionate about helping 
students succeed, it is tempting 
to teach all or many of the 
essential strategies at once. 
But this may mean nothing is 
mastered and little changes. 
The first two cohorts proved 
this even though the intention 
was to focus by limiting each 
school to selecting six to eight 
practices to implement per 
year. Six to eight practices 
proved to be too much. 


For Cohort 3, the focus was 
reduced to just two areas 

(e.g. two principles each 

in reading and math), and 
teacher buy-in and fidelity 

of implementation improved 
significantly. Even with the 
reduced focus, many teachers 
felt they were “drinking from 

a frehose”, especially if there 
were a significant amount of 
other initiatives going on in the 
district. It is equally important 
to stop doing certain things. As 
stated earlier, most teachers 
like learning and trying new 


approaches. The catch is that they want to try what is 


powerfully motivated teachers to commit to trying the 
new approach themselves. Then, when an effective 
principal invited a veteran teacher to share her positive 
experience with the strategy in front of the entire faculty, 


new without dropping what does not work. This leads to 
frustration (“there’s just too much to do ina school day”), 
confusion (“how do ‘leveled questions’ align with [fll 
in-the-blank] strategy”), or both. When deciding which 
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approaches to focus on, take the extra time to decide 
what approaches or strategies you will stop doing. 


° Get Specific. 

When learning a new principle, educators need 
specifics. Teaching a strategy as it applies to the 
average student or as if one-size-fits-all is not enough. 
Many teachers commented that they could see how the 
strategy applied to someone else’s class, but that they 
struggled to see how to use it 
in their own situation because 
of student levels, capabilities, 
subject matter, etc. While the 
general approach might be 
the place to start, itis crucial 
to help teachers see how 
they can adopt the strategy 
to different circumstances. 
This could involve repeating 
demonstrations of the 

same strategy in multiple 
situations, taking care to 
point out what changes and what does not based on 
the context. Furthermore, leaving educators with an 
entire presentation or manual does not specify the most 
important things to remember and do. Extensive teacher 
feedback indicated that many would have preferred to 
receive simple how-to guides or ‘cheat sheets’ that could 
be quickly referenced when preparing for or even in the 
middle of a lesson. 


¢ Acknowledge the Status Quo. 

Introducing a new approach fo instruction and the 
provision of student supports inevitably challenges 
common norms. Whether people stick with these norms 
due to inertia, habit, or simply lack of selfawareness, 
itis important to identify what needs to be changed 
and why. Some of this can be done in large-group 
settings. But in most cases, experience indicates that 
the best approach is through frequent, job-embedded 
observation accompanied by constructive, non- 
evaluative feedback. In most cases, educators who fail 
to quickly adopt a new strategy do so because they 
don’t realize how ingrained their standard approach 
really is. Pointing this out on an individual, realtime basis 
is one way to begin the improvement process. 


“It would have been nice to have a 
concrete outline of what the strategy 
looks like. There wasn’t really 
something concrete that | could walk 


away with. If | don’t know the names 
of the manipulatives, | can’t look them 
up to create them for my kids.” 
(Teacher) 


¢ Sweat the Small Stuff. 

Attention to detail matters. Participating teachers 

and leaders routinely praised the highly attentive 

and responsive MSM coaches, whom they saw as 
distinctively professional and supportive relative to 
other PD providers with whom they had interacted. 
Whether it was last-minute requests for additional 
materials or queries about new research, the coaches’ 
responsiveness increased participating educators’ 
willingness to engage with 
the changes MSM was 
encouraging. On the other 
hand, missing details had a 
negative effect. For example, 
in one case at least, not 
enough substitutes were 
secured in time to allow 

for pilot school teachers to 
attend a PD session at the 
demonstration school. As a 
result, the potential for impact 
was significantly diminished 
even though everything else was ready to go. 


¢ Combat the Tendency to Check the Box. 

Every new initiative runs the risk of oversimplification. This 
is especially true for efforts that attempt to distill complex 
ideas or insights into easy, practical application. When 
this happens, participants simply go through the motions, 
failing to use the recommended steps and core principles 
as a foundation to build upon. In addition to measuring 
progress and focusing on results rather than just effort, we 
have seen two additional antidotes. First, regularly return 
to this foundation. While checklists and quick tip guides 
are useful in the moment, it’s also important to remind 
participants of the why and how—where the strategies 
come from, why you've focused on this initiative above 
any other, and how these principles are intended to 

be used. And second, routinely promote progression. 
Encourage participants to build on their knowledge 

and expand their mastery to increasingly challenging 
situations. In doing so, they will need to remember the 
foundation while remaining engaged through additional 
growth. 


¢ Encourage Innovation, But Be Wary 

of Wandering. 
In the most recent school year, one high-performing 
school created their own program to support the 
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implementation of their selected MSM principles and 
practices. The MSM team was initially encouraged 
when they learned of the new program because the 
local team had designed it to fit their local context. 
However, as time passed it became obvious that much of 
the new program, while potentially effective, was not in 
fact based in rigorous research. It may be that this school 
is ahead of the research. But maybe not. The point is 
that it’s easy for local adaptations to veer quickly from 
appropriate innovations to problematic dilutions. And 
while it may not always be easy to tell the difference, 

the solution lies in returning to original principles (in this 
case, relying only on high-quality research) and being 
deliberate about treating the effort as an experiment, 
one that needs measurement to be proven productive. 


A CALL TO ACTION 


Looking forward, there are a number of areas in which 
school-based interventions designed to drive student 
success can improve. 


First, everyone interested in the success of middle 
school students can begin by becoming familiar with 
and using the resources made available through 

the Middle School Matters initiative. The research, 
frameworks, strategies, and materials available on 
www.greatmiddleschools.org are excellent sources for 
funders, educators, and researchers alike. 


Funders, whether private or public, play an outsized role 
in the nature and scope of efforts to transform school 
performance and drive student success. As such, they 
should recognize that providing additional training, 
resources, or capacity alone will not yield the change in 
individual or institutional behavior that needs to occur. 
Funders should consider whether and how they enable 
or discourage full adoption of new approaches. This 
reflection requires an understanding and application 

of insights from fields such as change management, 
implementation science, systems design—including 
many of the principles and lessons described above. 

It might also require additional funding for things like 
management training and capacity, measurement and 
accountability systems, and the development of tools 
and materials, all of which are traditionally viewed as 
undesirable overhead. Small but intentional investments 
in these areas will ensure that the rest of the funding 
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going to improve education outcomes achieves its goal. 


Educational leaders and policymakers at the state, 
district, and school levels should recognize and adopt 
at least two lessons from this experience. First, not all 
school training, initiatives, frameworks, or models are 
created equal. There is increasing evidence about what 
works and what does not when it comes to instructional 
practices and school environment. And while they may 
be plausible and occasionally effective, too many of 
the strategies educators know and practice are not 
based on the growing body of principles and practices 
that have been proven by research to improve student 
outcomes. It may not be easy, but educational leaders 
need to hold themselves to a high standard-limiting 
training and priorities to research-based approaches 
wherever possible. 


Second, educators can foster a culture where all 
educators are expected to learn, adopt, and master 
proven instructional strategies over the course of one’s 
career. While the MSM team has been encouraged 

by how many educators actively embrace change, 

there are still individuals who do not actively focus 

on deepening their personal practice over time. 

There are many reasons for this reality: misaligned 
incentives, philosophies held over from earlier eras, 
confusion about what really works, insufficient access 

to appropriate solutions, and occasionally individual 
resistance to change. However, the more educators are 
aware that research-based practices exist, the more they 
seek out and even demand access to these resources. In 
turn, they become more effective practitioners and their 
students will experience success. A culture of continuous 
improvement builds upon itself. 


Finally, change takes both time and focus. Once one 
knows what to do, itis tempting to try to do it all. But 
that’s a recipe for failure and disillusionment. Instead, go 
slow to go fast. Identify just a few priorities or initiatives 
that have been proven to succeed. Take the time to 
make sure those initiatives do, in fact, succeed before 
moving on to the next big thing. Doing this requires the 
patience, resources, and repetition for individuals to fully 
understand, implement, and eventually master new skills 
and behaviors. This will seem terribly inefficient at first. 
But after the foundation is in place and the stakeholders 
have seen positive results, the improvement process will 
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get faster and faster. 


Investing in effective middle schools to ensure students 
have every opportunity to succeed is important. Through 
the efforts of Middle School Matters and countless 
others’ work, there is now a growing body of knowledge 
and resources available for those who desire to 

help middle school students succeed. Achieving this 
goal is certainly challenging. But holding ourselves 
accountable, both individually and collectively, to a 
higher, rigorous standard of both instructional practice 
and effective implementation will generate untold 
benefits for educators and students around the country. 


CASE STUDIES 


The following two brief case 
studies help to illustrate this 
report's findings. 


Middle School A 
Middle School A identifed the 
following MSM principles'® on 
which it would focus: 

- Reading 2 — Teach 
word-meaning 
strategies within 
content area classes 

- Reading 4-Teach 
students to use 
reading 
comprehension 
strategies while 


reading i 

complex text other, that’s huge. 
- Reading 7 - (Principal) 

Maximize 


opportunities for 
students to read and connect 

a range of texts 

- Math 2 - Screen all students, using 
a universal screener, to identify those at risk 
for mathematics difficulties and provide 
interventions to students identified as being 
at risk. Monitor the development of 
mathematics knowledge and skills in 


' Refers to the number of the MSM instructional principle as listed in Table 1 above. 


“| have found that this whole process 
is transforming our school culture 

for breaking down barriers for those 
teachers who wanted to be left alone 
and didn’t want people to come into 
their classrooms. We are now hearing 
teachers soliciting feedback. That has 


been the shining star for the year—for 
teachers to offer help and accept help 
from colleagues. It’s about teachers 
eT} oliatom (releal-lecm-lale mee) |(-r-16[0[-c} 
helping colleagues. So when you see 
those people starting to help each 


identified students 

- Math 4 — Develop students’ conceptual 
understanding of mathematics and provide 
ample opportunities to improve procedural 
fluency 

- Math 9 — Discontinue using practices that 
are NOT associated with improved outcomes 
for students and teachers 


The PD delivered by the MSM team focused on the 
following reading and math instructional strategies: 
1. Increase student vocabulary through explicit 
teaching and vocabulary maps 
2. Increase reading comprehension through 
leveled questions 
3. Partner students according to reading levels 
A. Increase math 
comprehension through the 
use of manipulatives 


Following the summer 
conference, the MSM team 
provided four two-day, onsite 
training opportunities. Session 
1 consisted of an introductory 
visit in which expectations 
were set and the MSM team 
became familiar with teachers, 
leaders, and school structure. 
In Session 2, the MSM coach 
and researcher led a training 
session on leveled questions 
for three hours during the 

first afternoon for all of the 
school’s teachers. In the same 
site visit, another researcher 
led a training session on 
manipulatives for math and 
science teachers. The next day in their classrooms, 
teacher coaches from Middle School A modeled the 
strategies they learned the previous day and received 
feedback from the researchers. Subsequently, the 
teacher coaches scheduled times to either model or 
observe the remaining teachers apply the reading 
strategies in their own classrooms. Session 3 focused 
on introducing vocabulary maps. In this case, the coach 
observed as many of the teachers as possible applying 
the strategy in their own classrooms and provided 
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feedback to each as she went. 
All teachers attended the PD sessions during the MSM 
site visits. Substitute teachers covered classes for three 
hours so the teachers were free to attend. Math and 
science teachers received PD 
for both reading strategies and 
math strategies. Other core 
teachers received only the 
reading training. 


These training sessions were 
generally viewed as highly 
effective, especially compared 
to other PD teachers have 
received. One teacher's 
experience was shared by 
many others: “[The MSM staff] 
have been wonderful. They've 
been very responsive when we 
have questions, etc. We were 
able to calibrate and adjust as we've run into questions.” 


At the same time, teachers provided some feedback 
regarding the delivery of PD by the MSM team. The 
biggest frustration was that the training covers a lot of 
information in a short period of time; many teachers 

felt they were “drinking from a fire hose.” After teachers 
learned the strategies in an initial session, they 
expressed a desire to receive follow-up trainings on the 
same topic with multiple opportunities to practice the 
new skills. In addition, some teachers expressed a desire 
to understand the broader scope and arc of the program 
before focusing on individual pieces like a particular 
strategy or even an individualized observation session. 


Implementation Infrastructure 

In conjunction with its participation in the MSM 
program, on their own initiative, leaders at Middle 
School A established a new, two-part structure to 
increase the likelihood of a successful implementation. 
While MSM provided guidance surrounding 
implementation plans, Middle School A set up its own 
internal coaching structure after returning from the 
summer conference to ensure that implementation 
would work while MSM was not on campus. First, they 
organized an onsite team of ‘teacher coaches’ that 
provide observation and coaching to other teachers in 
their same grades and subject matter weekly through 


“The students are engaged more. 

We already do a lot to get them 
engaged. But using the manipulatives 
and the leveled questioning, which 

| like to do with partners, has gotten 


them more engaged. I’ve seen 
higher test scores, even within this 
year compared to before we started 
leveled questioning.” 

(Teacher) 


professional learning communities (PLCs). For example, 
all math and science teachers in a specific grade meet 
together weekly to talk about the curriculum and PD, 
including MSM strategies, that should be used to teach 
the curriculum that week. The 
onsite teacher coaches model 
lessons for the other teachers 
approximately once a month. 
They conduct observations 
and provide feedback to the 
other teachers so that each 

is getting MSM training. In 
addition, each teacher has 

a partner teacher in another 
subject area that they check in 
with regularly. Despite different 
subjects, partner teachers can 
discuss ideas for using MSM 
reading strategies because 
both apply them in their 
classes. 


The district’s role in MSM has been one of 
encouragement and observation but not of direct, 
additive involvement in developing capacity or ensuring 
adoption. The district staff regularly comes to Middle 
School A to observe the teachers implement the MSM 
strategies. The district also checks in periodically to see 
how MSM implementation is progressing, but there 

was no formal process for monitoring or measuring 
implementation from a district standpoint. 


While adoption and mastery of the MSM strategies 

is still on-going, leaders and teachers believe that the 
establishment of this implementation structure and 
accompanying learning practices among educators 
have been central to their success to date and, 
especially, any improvement they see in the future. As 
the Middle School A principal said: “| have found that 
this whole process is transforming our school culture for 
breaking down barriers for those teachers who wanted 
to be left alone and didn’t want people to come into 
their classrooms. We are now hearing teachers soliciting 
feedback. That has been the shining star for the year—for 
teachers to offer help and accept help from colleagues. 
It’s about teachers helping teachers and colleagues 
helping colleagues. So when you see those people 
starting to help each other, that’s huge.” 
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School District B 

School District B has a high rate of English-Language 
Learners and came to MSM seeking extra support to 
help these student succeed more quickly. The district, 
demonstration school, and pilot school leadership 
worked together to identify six reading MSM principles 
on which it would focus—three in reading and three in 


3. Guide students through reading activities 
using Rigorous Readers 


The first visit from the MSM team was primarily 
introductory and consisted of observations, introductions 
between MSM staff and school and district staff, 
discussions with school leadership about student 


writing. 

- Reading 2-Teach 
word-meaning 
strategies within 
content area classes 

- Reading 4-Teach 
students to use 
reading 
comprehension 
strategies while 
reading complex 


text 
- Reading 6—- Guide 
students during 


textrelated oral and 
written activities that 
support the 
interpretation, 
analysis, and 
summarization of 
text. 

- Writing 1 — Establish 
consistent 
schoolwide 
practices for using 
writing as a tool to 
support student 
learning in all 
content areas 


performance, and discussions about expectations for 
MSM delivery. Prior to the second visit, the MSM team 


“One of the things we did in the 
January PD session was provide 

a standards booklet for each of our 
students, and | had teachers 

read it with me. Then | modeled a 
Level 1 question to give them 

a reminder of what we’re trying to 
accomplish. Then | modeled 

a Level 2 and then a Level 3. | had 
each campus teacher practice 
developing the different leveled 
questions as a group. MSM sent 
us a video link of a teacher modeling 
Level 2 questioning, which 

gave me an idea of where it was 
going. All of the campuses | 
support receive this training.” 
(District Coordinator) 


sent website links to the district 
so they could access and 
begin using the questioning 
strategies. The district’s dean 
of instruction and district 
coordinators disseminated the 
information to the teachers 

by modeling the strategies 

in CLCs based on what 

they learned from the MSM 
materials and website links. 


Webinars were primarily 
used to deliver training on 
instructional strategies, while 
site visits were primarily 
focused on answering 
questions, modeling lessons, 
and providing feedback. 
Because it was supported 
by teams at both the district 
and school level (see below), 
this “flipped” approach to 
PD accelerated the pace of 
learning and coverage of 
different topics. In Session 


2, the MSM team answered questions about Level 1 


- Writing 2 — Explicitly and systematically teach 
students the processes, knowledge, and skills 
of effective writing 

- Writing 7 — Improve teacher capacity to 
teach writing and use it as a tool for learning 


The PD focused the following instructional strategies - 
stemming from those six principles - that were introduced 
at the summer conference: 
1. Increase student vocabulary through 
vocabulary maps and Frayer models 
2. Increase reading comprehension through 
leveled questions 


questioning, introduced Level 2 and 3 questioning, 
taught vocabulary with graphic organizers, and gave 
examples of how to teach students to write essay 
introductions. The PD for Level 2 and Level 3 questioning 
was delivered in a webinar prior to Session 3, which 
focused on leveled questioning, but training was 
primarily spent answering questions, modeling lessons, 
observing classes, and providing general feedback to 
groups of teachers in CLCs. 
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Implementation Infrastructure 

Relative to others participating in Cohort 3, School 
District B was unique in the role the district played 

in the implementation of MSM. Prior to beginning 

MSN, School District B had established a role, called 
district coordinators, to provide PD to principals and 
teachers. The coordinators are responsible for delivering 
curriculum and training to contentarea teachers through 
collaborative learning communities (CLCs), while school 
administration is ultimately 
responsible for enforcing 
MSM implementation. 


The district coordinators, who 
are each assigned fo a specific 
content area, visit schools 
weekly to model lessons, 
observe classes, and clarify 
how teaching strategies should 
be used. Since the district is 
responsible for engaging 
teachers in curriculum and 
associated training through 
district-led PD, each school 

in the district receives the same training. Building on 
this infrastructure and practice, School District B district 
coordinators attended MSM training and incorporated 
it into the PD they delivered to all middle schools in the 
district. 


In each school the teachers meet several times a week 
in CLCs by subject area. They receive training from the 
district coordinators in these meetings and share ideas 
regarding how to incorporate teaching strategies into 
the curriculum to best meet the needs of their students. 
This multi-level, integrated learning and implementation 
structure was highly effective for rolling out and 
adopting new initiatives such as MSM. 


The district coordinators and administrator of middle 
schools have regular meetings to analyze student data 
and determine which strategies-MSM or other—and 
curriculum to introduce, continue, or discontinue. 

The only area where the district keeps track of the 
effectiveness of the strategies is through discussions in 
CLCs, where department heads provide training and 
support for the teachers when the district coordinators 


“| think my students feel more 
confident. Even as adults, we 
sometimes find words we don’t know, 
but we use context clues to figure 

it out, but the students sometimes 


struggle with that process. But now 
they better see the clues, and they 
arent intimidated by words as much.” 
(Teacher) 


are not available. As a result, it is difficult for the teachers 
as well as school leadership to accurately measure how 
often or how well MSM strategies are implemented. 


Innovation or Departure? 

One of School District B’s primary areas of focus was 
on a strategy the district labeled Rigorous Readers. 
The primary purpose of Rigorous Readers was to help 
students analyze complex texts in a way that mimics 
the standardized exam. The 
Rigorous Readers strategy was 
created to address MSM’s 
Reading Principle 6: Guide 
students during text-related 
oral and written activities that 
support the interpretation, 
analysis, and summarization 
of text-as well as to combine 
several MSM strategies into 
one instrument. It includes 
both a reading and writing 
component, which allows 
teachers to evaluate student 
comprehension. Rigorous 
Readers was not prescribed by MSM as a research- 
based instructional strategy, but was adapted from 
another conference attended by district administrators 
after the MSM Summer Institute. 


While teachers and administrators felt it was an effective 
and innovative strategy, the Rigorous Readers strategy 
was broadly but incorrectly identifed as an MSM 
research-based strategy. On one hand, this type of 
innovation should be welcomed as the educators were 
adapting tools to their context. On the other hand, the 
identification of Rigorous Readers as an MSM strategy 
illustrates the potential for educators to depart from the 
high standard of relying exclusively on approaches 
proven by high-quality research. Going forward, 

both MSM and other efforts to improve middle school 
education will need to grapple with this challenge. 
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